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Abstract. Generating functions have useful applications in many fields of study. In this paper, 
the generating functions will be introduced and their applications in combinatorial problems, 
recurrence equations, and physics will be illustrated. 



1. Introduction. 

Working with a continuous function is sometimes much easier than working with a 
sequence. For example, in the analysis of functions, calculus is very useful. However, the 
discrete nature of sequences prevents us from using calculus on sequences. A generating 
function is a contimiotis ftmction associated with a given sequence. For this reason, 
generating functions are very useful in analyzing discrete problems involving sequences 
of numbers or sequences of functions. 

Definition 1-1. The generating function of a sequence {/n}^o defined as 

oo 
n=0 

for < R, and R is the radius of convergence of the series. 

It is important that the series has a nonzero radius of convergence, otherwise f{x) 
would be undefined for all x ^ 0. Fortunately, for most sequences that we would be 
dealing with, the radius of convergence is positive. However, we can certainly construct 
sequences for which the series (1-1) is divergent for all a; ^ 0; /„ = 71" is one such 
example. 

Example 1-2. As a simple example of a generating function, consider a geometric 
sequence, = a". Then 

f{x) = y a":c" = . (1-2) 

This series convergences absolutely whenever \ax\ < 1. Therefore, the radius of conver- 
gence is i? = l/\a\. 

For the rest of the paper, if not mentioned otherwise, x is always chosen to be small 
enough such that any series encountered in our analysis converges absolutely. Now we 
shall discuss an application of generating functions to linear recurrence problems. 
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2. From Recursion to Algebra. 

Generating functions can be used to solve a linear recurrence problem. 

Definition 2-1. The problem of linear recurrence is to find the values of a sequence 
{un} satisfying 

m 

CkUn+k = V, for some constant v and any integer n > 0, (2-1) 

fe=o 

given the initial values uo,ui, . . . , Um-i, where both cq and Cm are nonzero. 

In order to solve this recurrence problem, we use the following property of generating 
functions. 

Proposition 2-2. If {wn}^o sequence with a generating function u{x) and k 
is a positive integer, then 



71=0 L j=0 

Proof: We start from the definition of u{x). 



(2-2) 



OO k — 1 oo 

u{x) = J2 ^3^' = E "■r'^' + Yl "i^' • (2-3) 

j=0 j = j=k 

Shifting the summation index in the second term to n = j — k, we obtain 

k—l oo 

u{x) = Ujx' + ^ w„+fea;"+'=, (2-4) 

j=0 n=0 

from which Eq. (2-2) immediately follows. □ 
By multiplying both sides of Eq. (2-1) by a;" and summing from n = to oo, we find 

m oo oo 

XI Cfe X! = X x" = . (2-5) 

fe=0 n=0 n=0 

Using Proposition 2-2, each term on the left hand side with fixed k can be expressed in 
terms of (/(.r) and known constants wq, mi, . . . , Um-i- We have now reduced Eq. (2-5) 
into an algebraic equation for u{x), which can be easily solved. 

After finding u{x), we write down the Taylor series expansion of u{x) around x = 0. 
Because the Taylor series of a fimction is unique (if it exists), the coefficient of .t" in 
the Taylor series must be zt„. To illustrate this method, we shall use it on the following 
example from physics. 

Example 2-3. In special relativity, the usual one dimensional velocity addition for- 
mula v' = u + v is modified into [1, p. 127] 

^' = ^, (2-6) 

I +UV 

with v', u, and v measured in units of the speed of light c. We will use this velocity 
addition in the following problem. Suppose there are infinitely many cars labeled by 
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integers n > 0. The (n + l)-th car moves to the right relative to the n-th car with a 
relative velocity v {0 < v < 1). In our reference frame, we denote the velocity of the 
n-th car by u„. Assuming uo = 0, find u„ for all n > 1. 

Solution: Notice that is the addition of u„ and v using the addition formula 

in Eq. (2-6). 

Un+i = ^ , forn>0. (2-7) 

i + UnV 

This recurrence is not linear, and therefore we may not apply our previous method 
directly. With a little manipulation, however, this recurrence can be transformed into 
a linear recurrence. 



1 - u„ 



+1 



l+U„V~Un-V_ (1-M„)(1-W) 



1 



1 + UnV 
l+V 



Defining 



a 



1 - Un+l 
l + V 



1 + V ~ v{l — Un) 
1 V 



1 — V J 1 — Un 1 — V 

-, and /„ 



(2-8) 



1) ' • 1 7 — J IV -t 7 

— V 1 — V i- — Un 

Eq. (2-8) can be written as 

fn+l = afn - A, (2-9) 

which is a linear recurrence in /„. Now multiply both sides by and sum from n = 
to oo. 

oo oo oo 

-{m-fo)=af{x)--^, (2-10) 
a; ^ 1 — a; 

where we have used Proposition 2-2 to simplify the left hand side. The initial condition 

of /„ is given by /o = 1/(1 - Uo) = 1- Solving for f{x) yields 

1 Aa; 



m = 



1 — ax {1 — x){l — ax) 

1 1 



1 — aa; a — 1 1 — x 1 — ax 
Using the definitions of a and A, we find 

A 1 



a-1 2' 



and hence 



1 1 

+ 



1 — ax 1 — x 



oo oo 

{a- + l)x- = J2f^x-. 



(2-11) 



(2-12) 



(2-13) 



n=0 



Since f{x) is the sum of two geometric series, we conclude that the Taylor series 
around has a positive radius of convergence. Therefore, 2/„ = a"-|-l by the uniqueness 
of Taylor series, and 

1 a" - 1 , , 

u„ = 1 - — = . (2-14) 
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Since a > 1, we conclude that < m„ < 1 for all n > 1. Physically, this result shows 
that any car moves with a speed less than c (remember that we are writing u„ in units 
of the speed of light). Special relativity predicts that any massive object always travels 
slower than light [4, p. 119]. □ 

3. Applications to Combinatorial Problems. 

Many combinatorial problems can be solved with the aid of generating functions. In 
particular, let's consider the problem of finding the number of partitions of a natural 
number. 

Definition 3-1. [6, p. 169] A partition of a natural number n is a way to write n as 
a sum of natural numbers, without regard to the ordering of the numbers. 

Example 3-2. 1 + 1+ 3 + 1 is a partition of 6. 

With this definition, the generating function of the number of partitions of n has a 
simple form. 

Theorem 3-3. [6, p. 169] Ifpn is the number of partitions of n and po = 1, then 



E^^"-" = n (3-1) 

n=0 fe=l 

Proof: First we need to establish the convergence of the infinite product for x ^ 0. 

oo 

This infinite product converges absolutely if the series ^a;*^ converges absolutely [3, p. 

53]. Thus, the right hand side converges absolutely for |a;| < 1. 

Each factor in the infinite product can be expressed as a geometric series. 



fe=l k=l '-rn=0 



(3-2) 



In this form, we can see that the coefficient of the term is equal to the number of 

oo 

ways to choose integers {?Tij.|TOfe > 0, fc = positive integers} satisfying n = ^^m^fc. If 

k=l 

we take the x™'''^ term from the A;-th factor, then we obtain a;". By comparing this with 
Definition 3-1, we conclude that the coefficient of a;" is equal to the number of partitions 
of n. □ 

As a check, let's try expanding the right hand side of Eq. (3-1) up to a;^. 

oo 

^ p„a;" = (1 + X + x2 + + x^)(l + + x^){l + x^)il + x^) + ■ ■ ■ 

n=0 

= (1 + X + + x^ + x'')(l + x^ + + 2x^) + • • • 
= 1 + X + 2x2 + Sx^ + 5x^ + O(x^), 

Pi = 1, P2 = 2, P3 = 3, P4 = 5. (3-3) 

We can easily verify that Eq. (3-3) correctly gives the number of partitions of 1 to 4. 

Another important combinatorial problem that can be easily solved with generating 
functions is Catalan's problem [6, p. 260]. 
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Example 3-4 (Catalan's Problem). Given a product of n letters, how many ways can 
we calculate the product by multiplying two factors at a time, keeping the order fixed? 
As an example, for n = 3, there are two ways: (ai02)a3 and 01(0203). 

Solution: Denote the solution for n m by Km- It is clear that K2 = 1. For later 
convenience, we define Ki = 1. Now consider the n = m + 1 case. Suppose at the 
last step of the multiplication, we have be, where b — ai ■ ■ ■ aj, c = Oj+i ■ ■ ■ Om+i, and 
1 < j < m. Notice that there are Kj ways to multiply the factors in b, and there are 
Km-j+i ways to multiply the factors in c. Thus, for a given j, there are KjKm-j+i 
ways to multiply ai • • • Om+i- The total number of ways can be obtained by summing 
over all possible values of j. 

m 

Km+i = ^KjKm-j+i, m>l. (3-4) 
Multiply both sides by a;"*+^ and sum from m = 1 to m = 00. 

00 00 m 

J2 Km+ix^+' = E E KjKm-j+ix^+'. (3-5) 

m— 1 m— 1 j—1 

As usual, we define the generating function for {i^nj^i- 

00 

/^(a;) = ^if„ar". (3-6) 

The left hand side of Eq. (3-5) is 

00 00 

^ Km+lX"'+^ = ^ Kr^x" - K,x = K{x) - X. (3-7) 

m=l n=l 

Now consider the expression for K{xY. 



00 



K{xf = J2 KjX^ Kix\ (3-8) 

Let i = m, — j + l, where m > 1. For a given m, j can be any integer from 1 to m, since 
i>l. Thus, we can rewrite Eq. (3-8) as 

00 m CO m 

K{xf = E E K^K^.i+ixix^-i+' = E E KjKrn-j+ix^+'. (3-9) 

m=l j=l m=l j=l 

By using Eqs. (3-7) and (3-9) in Eq. (3-5), we obtain a quadratic equation for K{x). 



K{xf - K{x) +x = 0, or K{x) = i ± \s/l-Ax. (3-10) 

Notice from Eq. (3-6) that K{0) = 0, which means we must take the negative sign for 
the square root. 

Kix) = I^VTB^. (3.11) 
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It is clear that the square root has a converging power series around a; = for |x| < 
and hence the infinite series defining K{x) has a radius of convergence of j. 
Use the binomial formula to obtain the power series expansion of Vl — 4a;. 

^ = I + V (|)(i-l)(i-2)-(i-n-M) 



= l-2x 



f^(^l)n-i 1 • 3 • 5 • • ■{2n - 3) 



n=2 



= 1 - 2a; - 2 ^ 



n=2 



1 ■ 2 ■ 3 ■ ■ ■ (2n - 2) „ 

n!(n-l)! ^ 



n—l ^ ^ 



1 /2(n- 1) 
n\ n—l 

Now we substitute Eq. (3-12) into Eq. (3-11) to find 

1 /2(n- 
n\ n—l 



n=l ^ 

1 /2m^ 



n=l n=l 

or 

= ("""), m>0. (3-14) 

TO -|- 1 \ TO y 

This problem was solved by Catalan in 1838 [6, p. 259-260], and the Catalan numbers 
are conventionally defined as C„ = Kn+i, for n > 0. □ 

There are many other applications of generating functions in combinatorial problems 
that cannot be covered here. A wide variety of examples are discussed in [5, Chapter 
3]. 

4. Legendre Polynomials. 

So far, wc have only discussed generating functions of sequences of numbers. However, 
in Section 1, 1 mentioned that generating function methods can also be used to analyze 
sequences of functions. One interesting example is the generating function of Legendre 
polynomials. As we shall see, the generating function provides a physical insight, with 
a deep connection to electromagnetism. 

There are several ways to define the Legendre polynomials P„(t). For example, 
they can be defined as solutions to a differential equation [2, p. 96]. For our purposes, 
however, it is more convenient to define Pn(t) as follows. 

Definition 4-1. The Legendre polynomials {^n(i)}«^o defined in the interval 
— l<t<l. [2, p. 100] They satisfy the recurrence relation 

{n + l)Pn+i{t) = {2n + l)tPn{t)-nPn-i{t) forn>l, (4-1) 

with Po(i) = 1 and Pi{t) = t. 

From this definition, it is easy to prove by induction that P„ (t) is a polynomial in t 
of degree n. We now want to find the generating function of Pn{t). In order to avoid 
confusion in the notation, we denote the generating function of P„(t) for fixed t as 

gt(x) =^P„(t):r". (4-2) 

n=0 
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Taking the derivative gives 

oo 

xQ't{x) = J2nPn{t)x^. (4-3) 

71=1 

Let's multiply Eq. (4-1) by and sum from n = 1 to n = oo. 

oo oc 

^(n+l)P„+i(t)x"+i = ^[(2n+l)tP„(t)-nP„_i(t)]x»+i 

n=l n=l 

oo oo oo 

J2 mPm{t)x'" = 2tx J2 nPn{t)x'' + a;^ ^ [tPk+i{t)x'' -{k + l)Pk{t)x''] . (4-4) 

m=2 n=l k=0 

The last step follows from the substitutions m = n+1 and k = n — 1. 
We note from Eqs. (4-2) and (4-3) that 



J2 mPm{t)x'^ = xQ'tix) - Pi{t)x = xQ'tix) - tx, (4-5) 

OO 

Y,{k + l)Pk{t)x'' = xQ'tix) + Qt{x), (4-6) 



m=2 



fe=0 

while Proposition 2-2 implies 



oo 

X 
fe=0 



J2Pk+iit)x'' = Qt{x) - Po{t) = Qt{x) - 1. (4-7) 



Thus, Eq. (4-4) simpUfies to 

xQ't{x)-tx = 2tx'^Q[{x)+tx{Qt{x) - l) - x'^ {xQ[{x) + Qt{x)) , 



By integrating Eq. (4-8) and imposing the initial condition Qt{0) = Po(t) = 1, we obtain 

oo 

Qt{x) = Pn{t)x- = (4-9) 
Vl-2te + x^ 

To find the radius of convergence of the power series of Qi{x), wc need to find the 

location Zg (in complex plane) of the singularity nearest to the origin. Qt{zs) is singular 
if 

1 - 2tZs + zl = Q, (4-10) 

Zs = t±Wl-f^, \Zs\ = l. (4-11) 

Therefore, Q{z) is analytic in the region \z\ < 1, and its power series converges absolutely 
in this region. 

In electrostatics, the potential along the z axis duo to an azimuthally symmetric 
volume charge; distribution p{r,9) is given by [2, p. 35] (we set 47reo = 1) 

V{z) = 27T r dr'r'^ f dO' sinO' , ^^''''^'^ (4-12) 
Jo Jo V ^2 _ 2zr' cos 6' + r'^ 
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If p{r',9') is bounded, p{r',9') = for r' > a, and we are only interested in V{z) for 
z > a, then 

f" r'^ r o(r' 6') 

V{z) = 27T dr'— de'sme'^= i^ ' ' (4-13) 
Jo z Jo Vl-2a;cos6i' + a;2 

where x = r' /z < 1. 

Now we may use Eq. (4-9) with t = cosO' because |cos^'| < 1. Eq. (4-13) can then 
be written as 

oo 

Jin 



with 



qn = 2TT dr'{r')''+'^ d0' sin 0' p{r' ,0')Pn{cos 9') 
Jo Jo 

qn = j d^r'r"'p{r',9')Pn{cos0'). (4-15) 

The numbers are called the multipole moments. In particular, qo is the monopole 

moment (or total charge), and qi is the dipole moment [2, p. 146]. Since the n-th moment 
term in the potential falls off as 1/2:"+^, the first nonzero moment qn characterizes the 
behavior of V{z) as z/a^ 00. 

We can see that the generating function of P„(t) appears naturally in electromag- 
netism. This technique of expanding the potential as a series of "moments" is very 
useful, and is called "multipole expansion" . 

An important property of P„ (t) can be shown directly from the generating function 
by considering Qt{—x). 

00 ^ 

Notice that the right hand side is also equivalent to Q-t{x). 

00 00 

J2{-^rPn{t)x- = Pn{-t)x\ (4-17) 

n=0 n=0 

From the uniqueness of the power series of Qt{—x), we obtain P„(— i) = (— l)"P„(t). 
Therefore P„(t) is an odd (even) polynomial if and only if n is odd (even). 
Another property can be obtained by setting t = 1 in Eq. (4-9). 

00 - 00 

y p„(i)x" = , = - — = y ,x". (4-18) 

n—O ^ n—0 

Thus, Pn(l) = 1 for all n. Since Pn{t) is odd for odd n, we also obtain P„(— 1) = (—1)"". 
5. Useful Trick to Find a Generating Function. 

In Section 2, we saw that we can easily find the generating function of a sequence if 
that sequence is defined through a linear recurrence. However, in some cases, we may 
not have a linear recurrence, such as in the Catalan's problem in Section 3. For some 
sequences without a linear recurrence, it is possible to obtain the generating function 
using a convolution property. In fact, we have actually used this property to solve the 
Catalan's problem. 



Generating Functions and Their Applications 



9 



Definition 5-1. A convolution of two sequences {fn}^=o ^^'^ {9n}^=o is another se- 
quence denoted by {(/ * g)n}^=o, with 

n 

if *9)n = Y^ fkgn-k- (5-1) 

fe=0 

Theorem 5-2. Let {un}^=o and {wn}55°,o two sequences with generating func- 
tions u{x) and v{x) respectively. If Wn = {u* v)n and w{x) is the generating function 
of {wn}^^o, then 

w{x) = u{x)v{x). (5-2) 

The radius of convergence of w{x) is the minimum of the radii of convergence of u{x) 
and v{x). 

Proof: Let r > and s > be the radii of convergence of u{x) and v{x). Denote 
t = min(r, s). Consider the product 

oo oo 

u{x)v{x) = ^'^UiX^^'^VjX^ , (5-3) 

for I a; I < t. Since both series converge absolutely, we may rearrange the terms in the 
double summation. Suppose we want to group the same powers of x. We can do this by 
writing j = n — i, with n > 0. For each n, i goes from to n, because j is nonnegative. 

oo n oo 
u{x)v{x) = UiX'^Vn-iX^~^ = WnX^. (5-4) 

n=0 i=0 n=0 

This is precisely the definition of w{x), and the series on the right hand side converges 

absolutely for |a;| < t. □ 

As we shall see later in this section, it is sometimes more convenient to find a 
generating function for {a„/n!} instead of {a„}. This is the motivation to define the 

exponential generating function. 

Definition 5-3. The exponential generating function F{x) of a sequence {/n}J^o 
defined as 



The exponential generating functions have the following property. 

Lemma 5-4. If F{x) is the exponential generating function of a sequence {/nj^o? 
then 



J2fn+i-=F'{x). (5-6) 

n=a 

Proof: Differentiate both sides of the definition of F{x) in Eq. (5-5). 

F'{x) = Yl ™/™^ = E /^T^-Tm = E/»+i^- (5-7) 
ml ^-^^ (m— 1)! n! 



T=l ^ ' n=0 

□ 
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We can state a theorem analogous to Theorem 5-2 for exponential generating func- 
tions. 

Theorem 5-5. Let {/„}J^Lo '"^'^ {ffnli^Lo sequences with exponential gener- 

ating functions F{x) and G{x) respectively. If 

K = Y^ fk9n-k, (5-8) 

and H{x) is the exponential generating function o/{/i„}^oj ^'^^'^ 

H{x) = F{x)G{x). (5-9) 

The radius of convergence of H{x) is the minimum of the radii of convergence of F{x) 

and G{x). 

Proof: The proof follows the same steps as the proof of Theorem 5-2, by substituting 
Un = fn/n\ and Vn = gn/n\. □ 

We shall now discuss an example to illustrate the convolution method in a problem 
where the exponential generating function is a more convenient choice. 

Example 5-6. (Bell numbers) [6, p. 167]. Denote by 6„ the number of ways to write 
a set of n distinct elements as a union of disjoint subsets, with bo = 1. For n = 2, there 
are two ways: {01,02}, and {ai} U {02}. Find a formula for 5„. 

Solution: First we need to find a recurrence relation for 6„. Consider a set A of 
(n -|- 1) distinct elements, A = {ai,a2, ■ ■ ■ ,a„+i}. Suppose ai is contained in the first 
subset along with j other elements, where < j < n. There are (") ways to form this 
subset, which is the number of ways to pick j elements from {02, 03, • • • , On+i}- Once 
the first subset is fixed, we are left with a set S containing (n — j) distinct elements. 
There are bn-j ways of partitioning 5 into subsets, and therefore we may write 

^"+i=E(-)^"-=E(I)^'^' (5-10) 

by using k = n — j and (^) = If we define a sequence = IjJJLo) then its 

exponential generating function T{x) is given by 



^(^) = E = (5-11) 



ra=0 



Eq. (5-10) can be written as 



Wn = 

fe=0 



^(jM„-fc, (5-12) 



with Wn = 

Notice the similarity between Eqs. (5-12) and (5-8). Applying Theorem 5-5 on {6„}^q 
and {tn}^=Q, we obtain 

B{x)T{x) = W{x) 

00 ^ 00 ^ 

B(a;)e- = Vu;„^ = ^6„+A, (5-13) 

n=0 n=0 
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where 



n=0 



(5-14) 



is the exponential generating function of {&„}^o- 

According to Lemma 5-4, the right hand side of Eq. (5-13) can be written as B'{x). 
Thus, 

B'{x) = e^'Bix). (5-15) 
We can integrate Eq. (5-15) and use -B(O) = 6o = 1 to find 

1 



B{x) = e 



= - e 

e 



(5-16) 



Since has a power series that converges everywhere, we conclude that B{x) has an 
infinite radius of convergence. Let's write the power series expansion of B{x). 



1x1 

Bix) = -e = - 
e e 



, p'^-*' 



fe=i 



A;! 



e fc! n! 

n=l fe=l 



e (fc- 1)! n! 

n=l fe=l ^ 



Therefore, for any natural number n. 



_ 1 ^ fc"-i 
^"-el^(fc-l)r 



(5-17) 



(5-18) 



To check our answer, take n = 2. One way to find 62 is to sum the infinite series in 
Eq. (5-18). However, there is a simpler way if we notice that 



S"(x) = ^n(n-1)6„^ 



n=2 



r.n-2 



(5-19) 



and thus 62 = -B"(0). We can find B"{x) by differentiating Eq. (5-16) twice. 

B"{x) = (1 + e^)e«'+^-^ (5-20) 



Therefore, 62 = 2 as expected. Incidentally, by using Eq. (5-18) for n = 2, we have 
proven the following infinite series. 



E 



k 



= 2e. 



(5-21) 
□ 



Conclusions 

We have discussed some basic applications of generating functions, as a method to solve 
a linear recurrence or combinatorial problems. However, there arc certainly many more 
aspects in the subject that are not discussed here. Readers interested to learn more are 
invited to read [5] for a very extensive treatment of generating functions. 
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